Stochastic Approximation: Theory and Applications

By Prof. Gugan Chandrashekhar Mallika Thoppe | IISc Bangalore

Learners enrolled: 654 | Exam registration: 43

ABOUT THE COURSE:

Stochastic approximation refers to a class of iterative algorithms that can locate zeroes or optimal points of functions in scenarios where the function evaluations are compromised by noise. These algorithms are prominently utilized in regression, system identification, adaptive control, and increasingly in reinforcement learning and machine learning. This course will explore the design, theoretical convergence, and convergence rates of these algorithms, with a particular emphasis on applications in reinforcement learning

INTENDED AUDIENCE: UG, Masters and Ph.D. students in Computer Science and Engineering/Electronics and Communication Engineerings/Mathematics

PREREQUISITES: Real analysis, Measure-theoretic Probability, Optimization, Design and Analysis of Algorithms, Ordinary Differential Equations

INDUSTRY SUPPORT: Google Research, Microsoft Research, Adobe Research

Summary

Course Status :	Ongoing
Course Type :	Elective
Language for course content :	English
Duration :	12 weeks
Category :	Computer Science and Engineering Communication and Signal Processing Foundations of Computing Data Science
Credit Points :	3
Level :	Postgraduate
Start Date :	21 Jul 2025
End Date :	10 Oct 2025
Enrollment Ends :	04 Aug 2025
Exam Registration Ends :	22 Aug 2025
Exam Date :	25 Oct 2025 IST
NCrF Level	4.5 — 8.0

Note: This exam date is subject to change based on seat availability. You can check final exam date on your hall ticket.

Page Visits

Course layout

Week 1: Introduction to stochastic approximation: Motivating examples from web-crawling and reinforcement learning

Week 2: Conditional expectation: Examples, Definition, Least-squares-best predictor, Existence, Properties

Week 3: Martingales: Filtration, Adapted Process, Definition, Examples, Convergence

Week 4: Ordinary differential equations: Existence and Uniqueness of solutions, Gronwall inequality, Asymptotic Behaviors, Invariant sets, Internally chain transitive sets

Week 5: Convergence of stochastic approximation algorithms: ODE method

Week 6: Convergence of stochastic approximation algorithms: ODE method (continue.)

Week 7: Convergence rates of linear stochastic approximation algorithms

Week 8: Stability of stochastic approximation algorithms

Week 9: Two-timescale stochastic approximation: Convergence

Week 10: Stochastic Recursive Inclusions: Convergence

Week 11: Applications to reinforcement learning

Week 12: Applications to reinforcement learning (continue.)

Books and references

1. Borkar, Vivek S. Stochastic approximation: a dynamical systems viewpoint. Vol. 48. Springer, 2009.
2. Grimmett, Geoffrey, and David Stirzaker. Probability and random processes. Oxford university press, 2020.
3. Williams, David. Probability with martingales. Cambridge university press, 1991.
4. Borkar, Vivek S. Probability theory: an advanced course. Springer Science & Business Media, 2012.
5. Hirsch, Morris W., Stephen Smale, and Robert L. Devaney. Differential equations, dynamical systems, and an introduction to chaos. Academic press, 2013.

Instructor bio

Prof. Gugan Chandrashekhar Mallika Thoppe

IISc Bangalore

Prof. Gugan Thoppe is an Assistant Professor in the Computer Science and Automation (CSA) department at the Indian Institute of Science since 2019. He is also an Associate Researcher at the Robert Bosch Centre, IIT Madras. He received his Ph.D. in 2016 from the Tata Institute of Fundamental Research (TIFR), Mumbai. He then completed postdoctoral research at Technion Institute of Technology, Israel (2015-17) and Duke University, USA (2017-19). His research is supported by the CEFIPRA Indo-French grant, the Walmart CSR grant, and the DST-SERB's core research grant. He is also involved in collaborative research with NPCI, India's national payments corporation. He has won the Pratiksha Trust's Young Investigator Award, the IISc Award for Excellence in Teaching, the TIFR Award for the best Ph.D. thesis, and two IBM Ph.D. fellowships. His research interests include reinforcement learning, online learning, stochastic approximation, and random topology.

Course certificate

The course is free to enroll and learn from. But if you want a certificate, you have to register and write the proctored exam conducted by us in person at any of the designated exam centres.

The exam is optional for a fee of Rs 1000/- (Rupees one thousand only).

Date and Time of Exams: October 25, 2025 Morning session 9am to 12 noon; Afternoon Session 2pm to 5pm.

Registration url: Announcements will be made when the registration form is open for registrations.

The online registration form has to be filled and the certification exam fee needs to be paid. More details will be made available when the exam registration form is published. If there are any changes, it will be mentioned then.

Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc.

CRITERIA TO GET A CERTIFICATE

Average assignment score = 25% of average of best 8 assignments out of the total 12 assignments given in the course.
Exam score = 75% of the proctored certification exam score out of 100

Final score = Average assignment score + Exam score

Please note that assignments encompass all types (including quizzes, programming tasks, and essay submissions) available in the specific week.

YOU WILL BE ELIGIBLE FOR A CERTIFICATE ONLY IF AVERAGE ASSIGNMENT SCORE >=10/25 AND EXAM SCORE >= 30/75. If one of the 2 criteria is not met, you will not get the certificate even if the Final score >= 40/100.

Certificate will have your name, photograph and the score in the final exam with the breakup.It will have the logos of NPTEL and IISc Bangalore .It will be e-verifiable at nptel.ac.in/noc.

Only the e-certificate will be made available. Hard copies will not be dispatched.

Once again, thanks for your interest in our online courses and certification. Happy learning.

- NPTEL team